Patches in Vision

This special issue contains extended versions of the best papers of the two " Beyond Patches " workshops we ran in 2006 and 2007 IEEE Conferences on Computer Vision and Pattern Recognition (CVPR). In addition, some specially solicited papers have also been included which were not part of these two workshops but do highlight and reinforce the motivation and philosophy of these workshops. We refer to a " patch " agnostically as an ensemble of spatially adjacent pixels/descriptors which are treated collectively as a single primitive. Patches fall between the two extremes of individual pixels/descriptors and whole objects/images. Analyzing an image or video sequence in terms of patches, rather than individual pixels/descriptors, and so forth. Additional novel applications explored in this special issue include image restoration, image compression, pixel motion, and scene recognition. Our workshops and this special issue have been motivated by the almost ubiquitous employment of " patches " in recent years across the vision the community. The papers included in this special issue touch upon many of the benefits of patch-based representations in vision, image, and video processing. Gupta and Huang proposed a unique approach to image restoration that leverages a multilayer " patch-based " graphical model which unifies the low-level vision task of restoration and the high-level vision task of recognition in a cooperative framework. In their approach, they modeled images as MRFs over a patch-based representation. Through the incorporation of two spatial domain methods, they argue that it is possible to move toward the idea that high-level concepts like recognition can be used to aid low-level operations like restoration. To validate this argument, they introduce a transformed domain method analogous to the spatial domain patch-based MRF and implement the system for removing compression artifacts from images and videos. Chandler et al. demonstrate a unique method for measuring the capacity of natural image patches for visual masking. Their central thesis is that the current state-of-the-art models of visual masking have been optimized for artificial targets placed upon unnatural backgrounds. To circumvent this problem, they (i) measure the ability of natural-image patches in masking distortion, (ii) analyze the performance of a widely accepted, standard masking model in predicting these data, and (iii) report optimal model parameters for different patch types (textures, structures, and edges). A robust algorithm for subpixel motion estimation is proposed by El Mehdi et al. In the work entitled " A Robust Sub-Pixel Motion …